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CLAIMS: We claim: 

1 . A robust apparatus and method for visually tracking an object in three dimensions, 
comprising steps of 

(a) capturing the object with two or more imaging sensors; 

(b) maintaining a large number of 3D target location hypotheses; 

(c) projecting each target location hypothesis from 3D space to 2D image spaces of 
said imaging sensors; , 

(d) measuring in images of captured said imaging sensors, confidences about the 
presence of said target object; 

(e) combining said measured confidences of said hypotheses to obtain 3D location 
of said target object. ,p 

2. The method according to claim 1 , wherein the said imaging sensors are color 
cameras. 

3. The method according to claim 1 , wherein the apparatus is a computer, comprising; 
a video capture apparatus, memory and a processor. 

4. The method according to claim 2, wherein said color cameras are IEEE 1394 
cameras. 
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5. The method according to claim 1 , where projecting target locations is performed with 
projections that are obtained by calibrating said video sensors with respect to a 
reference coordinate system. 

6. The method according to claim 1 , wherein measuring confidences is performed 
based on color and motion cues. 

7. The method according to claim 6, wherein color cues are calculated by using a color 
model of the target object. 

8. The method according to claim 6, where the color model of the target is represented 
by a histogram that is estimated by collecting color samples of the target object. 

9. The method according to claim 6, wherein motion cues are calculated by calculating 
differences between images captured sequentially by the said imaging sensors. 

10. The method according to claim 1, wherein maintaining 3D target location hypothesis 
is performed by creating at each time step a set of 3D target location hypotheses. 

1 1 .The method according to claim 10, wherein said 3D target location hypotheses are 
created based on known 3D target location hypotheses from a previous time step. 
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12. The method according to claim 10, wherein said 3D target locations are initially 
distributed randomly in the space viewed by said imaging sensors. 

13. The method according to claim 1 , where maintaining 3D target location hypotheses 
involves adding random displacements to said location hypothesis at each time step. 

14. The method according to claim 1 , where combining said measurements involve 
calculating the average of the locations of said 3D target location hypotheses. 

1 5. The method and apparatus according to claim 1 , where said target is a human 
appendage. 

16. The method and apparatus according to claim 1 , where said target is a human head. 



22 



